The present invention relates to security analysis and/or syntax testing of hardware and software and automated generation and execution of test cases.
Computerized communication, whether it occurs at the application level or at the network level, generally involves the exchange of data or messages in a known, structured format (a “protocol”). Specifically, a protocol refers to what is being communicated (for example, the data or message content). Types of protocols include, for example, networking protocols (including network packets), application program interfaces (APIs; including API calls, remote method invocation (RMI), and remote procedure call (RPC)), and file formats.
Software applications and hardware devices that rely on these formats can be vulnerable to various attacks that are generally known as “protocol abuse.” Protocol abuse consists of sending messages that are invalid or malformed with respect to a particular protocol (“protocol anomalies”) or sending messages that are well-formed but inappropriate based on a system's state. Messages whose purpose is to attack a system are commonly known as malicious network traffic.
Various systems have been developed that identify or detect attacks when they occur. This functionality, which is known as intrusion detection, can be implemented by a system that is either passive or active. A passive intrusion detection system (IDS) will merely detect an attack, while an active IDS will attempt to thwart the attack. Note that an IDS reacts to an actual attack. While an IDS might be able to detect an attack, it does not change the fact that an attack has occurred and might have damaged the underlying system.
A proactive solution to the attack problem is to analyze a system ahead of time to discover or identify any vulnerabilities. This way, the vulnerabilities can be addressed before the system is deployed or released to customers. This process, which is known as “security analysis,” can be performed using various methodologies. One methodology for analyzing the security of a device-under-analysis (DUA) is to treat the DUA as a black box. Under this methodology, the DUA is analyzed via the interfaces that it presents to the outside world. As a result, it is not necessary to access the source code or object code comprising the DUA.
For example, a security analyzer executes a test case by sending one or more messages (test messages) to the DUA. The analyzer then observes the DUA's response. A response can include, for example, registering an error or generating a message (response message). The DUA can then send the response message to the analyzer. Depending on the analysis test case being performed, the analyzer might send another test message to the DUA upon receiving the response message from the DUA. The test messages and response messages can be analyzed to determine whether the DUA operated correctly.
In one embodiment, a test message is “improper” in that its structure does not conform to the appropriate protocol. Protocol structure (also known as syntax) refers to the layout of a message, such as its fields, arguments, or parameters, and its possible length. In this embodiment, security analysis of the DUA would be similar to syntax-based vulnerability testing of the DUA's implementation of the protocol.
Each test case that is executed helps to analyze a different aspect of the DUA's security. Thus, in order to analyze the security of a DUA, it is necessary to execute several different test cases. While test cases can be created manually, it is more efficient to create them in an automated fashion (see, for example, U.S. application Ser. No. 11/514,809, filed on Sep. 1, 2006, entitled “Automated Generation of Attacks for Analyzing the Security of Communication Protocols and Channels” and U.S. application Ser. No. 11/745,338, filed on May 7, 2007, entitled “Modification of Messages for Analyzing the Security of Communication Protocols and Channels”).
In the past, a complete test case was created before the test case was used for testing. For example, consider a test case that calls for an exchange of messages: a first message (test message) sent to the DUA, a second message (response message) received from the DUA, and a third message (test message) sent to the DUA. In the past, the test case would define the first and third messages and cause them to be sent at the appropriate times. Since the third message was defined when the test case was created, it remained the same every time the test case was executed, regardless of the contents of the second message.
But what if the protocol called for the contents of the third message to depend on the contents of the second message? In this situation, the test case would have to parse the second message and then use this information to generate the third message. This would be difficult to achieve due to the dynamic nature of protocols, the ambiguity of protocol specifications, and the diversity of protocol implementations. As a result, static test cases can be insufficient to effectively perform syntax-based testing.